Popularization, multifunctionalization and downsizing of digital cameras bring new possibilities in the field of pattern recognition and media understanding. One of such possibilities is that an image acquired by a user is linked to any of various services. Such possibilities are unexceptionally present in the field of characters and documents. Intensive researches are conducted into camera-based character recognition and document/image analysis (see, for example, Non-Patent Documents 1 and 2). Particularly, interfaces utilizing digital cameras attached to mobile phones are important, and a variety of processes such as a character reading process and a translation process utilizing the interfaces are now under consideration (see, for example, Non-Patent Documents 3 and 4).
Prior art methods of retrieving image-based document/image data, i.e., a document and/or image, are as follows. In the Kauniskangas method, documents and/or images are each divided into paragraph regions and graphic regions, which are classified and expressed in a tree structure. For retrieval, the degrees of matching between a query and the respective regions of the documents and/or images in a database are determined, and an image having the highest matching degree is output as a retrieval result (see, for example, Non-Patent Document 5). Hull discloses a document indexing method and a retrieval method based on the number of characters of each word, and an image indexing method utilizing an invariant.
There is also disclosed a method, in which a text of a document is divided on a word-by-word basis, and the document is expressed by features defined by a sequence of the numbers of characters of the respective words. Features are preliminarily calculated for respective portions of a document in a database, and stored in a hash table. For retrieval of an input image, features are calculated for the input image in the same manner. The retrieval is achieved by accessing the hash based on the features of the input image and voting (see, for example, Patent document 1 and Non-Patent Document 6).
The methods described above deal with a high-resolution correct-orientation image obtained by a flat bed scanner or the like. Therefore, these methods cannot be employed for digital camera-based document/image retrieval which is dealt with by the present invention. The hull methods, for example, are based on the assumption that characters are separable in the input image. This assumption is not satisfied in the case of a lower-definition image or an image subjected to a geometric transformation such as a projective transformation which is dealt with by the present invention.    Patent document 1: JP-A-7(1995)-282088    Non-Patent Document 1: D. Doermann, J. Liang and H. Li, “Progress in Camera-Based Document Image Analysis”, Proc. ICDAR '03, pp. 606-616 (2003)    Non-Patent Document 2: K. Kise, S. Omachi, S. Uchida, M. Iwamura, “Current Status and Future Prospects of Camera-Based Character Recognition and Document Image Analysis”, Technical Report of the IEICE, PRMU2004-246 (2005.3)    Non-Patent Document 3: K. Yamada, S. Senda, “Ubiquitous Information Interface Using Mobile Camera”, Information Processing, 45, 9, pp. 923-927 (2004)    Non-Patent Document 4: Y. Watanabe, Y. Okada, Y-B. Kim, T. Takeda, “Translation Camera”, Proc. ICPR '98, pp. 613-617 (1998)    Non-Patent Document 5: K. Hannu, “Document Image Retrieval with Improvements in Database Quality”, Academic Dissertation of University of Oulu (1999)    Non-Patent Document 6: J. J. Hull, “Document Image Matching and Retrieval with Multiple Distortion-Invariant Descriptors”, Document Analysis Systems, pp. 379-396 (1995)